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Resume : Get article aborde le probleme de la detection de fautes dans les reseaux dyna- 
miques de type MANET. Les detecteurs de fautes non fiables fournissent des informations 
sur les processus defaillants. lis permettent de resoudre le consensus dans les reseaux asyn- 
chrones. Cependant, la plupart des detecteurs considere un ensemble connu de processus in- 
terconnectes par un reseau completement maille. Des telles hypotheses ne sont pas realistes 
dans les environnements dynamiques. Generalement, les implementations des detecteurs re- 
posent sur des temporisateurs dont les bornes sont particulierement difhciles a determiner 
dans le contexte des reseaux dynamiques. Get article presente une implementation asyn- 
chrone de detecteurs de defaillances adaptee aux environnements dynamiques. Nous prou- 
vons que notre algorithme permet d'implementer un detection de classe •OS' lorsque des 
proprietes sur la vitesse relative des transmissions et la connectivite sont satisfaites par le 
reseau sous-jacent. 
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Asynchronous Implementation of Failure Detectors with 
partial connectivity and unknown participants 

Abstract: We consider the problem of failure detection in dynamic networks such as 
MANETs. Unreliable failure detectors are classical mechanisms which provide information 
about process failures. However, most of current implementations consider that the network 
is fully connected and that the initial number of nodes of the system is known. This as- 
sumption is not applicable to dynamic environments. Furthermore, such implementations 
are usually timer-based while in dynamic networks there is no upper bound for communica- 
tion delays since nodes can move. This paper presents an asynchronous implementation of 
a failure detector for unknown and mobile networks. Our approach does not rely on timers 
and neither the composition nor the number of nodes in the system are known. We prove 
that our algorithm can implement failure detectors of class {^S when behavioral properties 
and connectivity conditions are satisfied by the underlying system. 

Key-words: failure detectors, distributed algorithms, dynamic networks 
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1 Introduction 

The distributed computing scenario is rapidly evolving for integrating unstructured, self- 
organizing and dynamic systems, such as peer-to-peer, wireless sensor and mobile ad-hoc 
networks. Nonetheless, the issue of designing reliable services which can cope with the high 
dynamism of these systems is a challenge. 

Failure detector is a fundamental service, able to help in the development of fault-tolerant 
distributed systems. Its importance has been revealed by Chandra and Toueg who proposed 
the abstraction of unreliable failure detectors in order to circumvent the impossibility result 
of the consensus problem in an asynchronous environment [FLP85llCT96| . Unreliable failure 
detectors, namely FD, can informally be seen as a per process oracle, which periodically 
provides a list of processes suspected of having crashed. In this paper, we are interested in 
the class of FD denoted OS*. Chandra and Toueg proved that by adding FD of class (}S 
to an asynchronous system, it is possible to deterministically solve the consensus problem 
(with the additional assumption that a majority of processes are correct). 

This paper focuses on FD for mobile and unknown networks, such as mobile ad-hoc 
networks (MANETs). This kind of network presents the following properties : (1) a node 
does not necessarily know all the nodes of the network. It can only send messages to its 
neighbors, i.e., those nodes that are within its transmission range0; (2) message transmission 
delay between nodes is highly unpredictable ; (3) the network is not fully connected which 
means that a message sent by a node might be routed through a set of intermediate nodes 
until reaching the destination node ; (4) a node can move around and change its transmission 
range. 

Most of current implementations of failure detectors are based on an all-to-all communi- 
cation approach where each process periodically sends a heartbeat message to all processes 
|LFA00l ISM01|, IPTOO]. As they usually consider a fully connected set of known nodes, 
these implementations are not adequate for dynamic environments for the reasons explained 
above. Furthermore, they are usually timer-based, assuming that eventually some bound 
of the transmission will permanently hold. Such an assumption is not suitable for dynamic 
environments where communication delays between two nodes can vary due to mobility of 
nodes. In [MMR03] . Mostefaoui et al. have proposed an asynchronous implementation of 
FDs which is timer-free. It is based on an exchange of messages which just uses the value 
of / (the maximum number of processes that can crash) and n (the number of nodes in 
the system). However, their computation model consists of a set of fully connected initially 
known nodes. Some recent works have been proposed which deals with the scalable nature 
of dynamic systems [LFAOO| IGCG01|. IBMSOSj . Nonetheless, few of them tolerate mobility 
of nodes [FTPS) ITTS04| and they are all timer-based. 

This paper presents a new asynchronous FD algorithm for dynamic systems of mobile 
and unknown networks. It does not rely on timers to detect failures and no knowledge about 
the system composition nor its cardinality is required. Yet, it has some interesting features 

^The concept of range models, for instance, homogeneous radio communication in MANETs. 
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that allow for scalability. The detection of process failures is based only on a local perception 
that the node has on the network and not on global exchanged information. 

The basic principle of our FD is the flooding of failure suspicion information over the 
network. Initially, each node only knows itself. Then, it periodically exchanges a QUERY- 
RESPONSE pair of messages with its neighbors, that is, those nodes from which it has received 
a message previously. Then, based only on the reception of these messages and the partial 
knowledge about the system membership (i.e., its neighborhood), a node is able to suspect 
other processes or revoke a suspicion in the system. This information about suspicions and 
mistakes is piggybacked in the QUERY messages. Thus, as soon as the underlying system 
satisfies an f-covering property^ suspicions and mistakes are propagated to the whole network. 
The f-covering property ensures that there is always a path between any two nodes of the 
network, in spite of / faults (/ < n) . 

Moreover, if the processes in the system satisfy some behavioral properties, our algorithm 
implements the failure detectors properties of the class Four behavioral properties have 
been defined. The membership property states that, in order to be known in the system, a 
node should interact (by sending messages) at least once with some others. The mobility 
property states that a moving node should reconnect to the network longtime enough in order 
to update its state regarding failure suspicions and mistakes. The responsiveness property 
states that after a given time, communication between some node in the system and its 
neighborhood is always faster than the other communications of this neighborhood. Finally, 
the mobility responsiveness property states that at least one correct node in the system does 
satisfy the responsiveness property and that its neighborhood is composed of non-moving 
nodes. 

The rest of the paper is organized as follows. Section [2] presents Chandra- Toueg's failure 
detectors. Section |3] defines the computation model. In Section 21 our asynchronous failure 
detector algorithm is presented considering that nodes do not move. Section [5] describes how 
the algorithm can be extended to support mobility of nodes. Simulation performance results 
are shown in Section while some related work are briefly described in Section [7] Finally, 
Section [S] concludes the paper. 

2 Chandra- Toueg's Failure Detectors 

Unreliable failure detectors provide information about the aliveness of processes in the 
system [CT96j . Each process has access to a local failure detector which outputs a list of 
processes that it currently suspects of having crashed. The failure detector is unreliable in 
the sense that it may erroneously add to its list a process which is actually correct. But 
if the detector later believes that suspecting this process is a mistake, it then removes the 
process from its list. Therefore, a detector may repeatedly add and remove the same process 
from its list of suspected processes. 

Failure detectors are formally characterized by two properties. Completeness characte- 
rizes its capability of suspecting every faulty process permanently. Accuracy characterizes its 
capability of not suspecting correct processes. Our work is focused on the class of Eventually 
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Strong detectors, also known as 'C><S'. This class contains all the failure detectors that satisfy 
(1) Strong completeness : there is a time after which every process that crashes is permanently 
suspected by every correct process ; (2) Eventual weak accuracy : there is a time after which 
some correct processes are not suspected by any correct process. 

3 Model 

We consider a dynamic distributed system consisting of a finite set 11 of n > 1 mobile 
nodes, namely, 11 = {pi, . . . ,Pn}- Contrarily to a static environment, in a dynamic system of 
mobile unknown networks, processes are not aware about 11 and its cardinality n. Thus, they 
know only a subset of processes in 11. There is one process per node and they communicate by 
sending and receiving messages via a packet radio network. There are no assumptions on the 
relative speed of processes or on message transfer delays, thus the system is asynchronous. A 
process can fail by crashing. A correct process is a process that does not crash during a run ; 
otherwise, it is faulty. Let / denote the maximum number of processes that may crash in the 
system {f < n). We assume that / is known to every process. To simplify the presentation, 
we take the range T of the clock's tick to be the set of natural numbers. Processes do not 
have access to T : it is introduced for the convenience of the presentation. 

The system can be represented by a communication graph G{V,E) in which yen 
represents the set of nodes and E represents the set of logical links. Nodes p, and pj are 
connected by a link {pi,pj) G iff they are within their wireless transmission range. In this 
case. Pi and pj are considered 1-hop neighbors, belonging to the same neighborhood. The 
topology of G is dynamic. Links are considered to be reliable : they do not create, alter or 
lose messages. Then, a message m broadcast by pi is heard by all correct processes in pi's 
neighborhood. Communications between 1-hop neighbors are either broadcast or point-to- 
point. 

When a node moves, we consider that it is separated from G. Afterwards, when it stops 
moving and reconnects to the network, it is reinserted to G. A node can keep continuously 
moving and reconnecting, or eventually it crashes. Nonetheless, a correct moving node will 
always reconnect to the network. A m,oving node is one that is separated from G and a 
non-moving node is connected to G. Let Pm be a moving node. We consider that Pm is not 
aware about its mobility. Thus, it cannot notify its neighbors about its moving. In this case, 
for the viewpoint of a neighbor, it is not possible to distinguish between a moving or a crash 
of Pm- During the moving, pm keeps its state, that is, the values of its variables. 

Definition 1. Range : In a network represented by G{V,E), rangci includes pi and the 
set of its 1-hop neighbors. In this case, \rangei\ is equal to the degree ofpt in G plus 1. Note 
that ranges are symmetric i.e. pi G range j => pj G ranges 

Definition 2. Range Density : In a network represented by G{V,E), the range density, 
namely d, is equal to the size of the smallest range set of the network : 

d = min{\rangei\),\lpi G 11 
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We assume that d is known to every process. 

Definition 3. f-Covering Network : A network represented by G{V, E) is f-covering if 
and only if G is (/ + l)-connected. 

By Menger's Theorem jYG98] . a graph G is (/ + l)-connected if and only if it contains 
(/ + 1) independent paths between any two nodes. Thus, removing / nodes from G leaves 
at least one path between any pair of nodes {pi,pj). Moreover, the range density d of the 
network will be greater than f + 1, d > f + 1. These lead to the following remark. 

Remark 1. Let G{V,E) be an f-covering network, thus there is a path between any two 
nodes in G, in spite of f < n crashes. 

4 Implementation of a Failure Detector of Class (^S 

This section presents a failure detector algorithm for a network where nodes do not 
move. The next section ([SJ extends this algorithm to support node mobility. This section 
firstly presents the principle of the query-response mechanism on which our algorithm is 
based. Then, it introduces some behavioral properties that, when satisfied by the underlying 
system, allow to implement a failure detector of the class OS'. Based on such a properties, 
we propose an asynchronous failure detection algorithm. A proof that our implementation 
provides a failure detector of class <>5 is also presented. 

4.1 Query- Response Mechanism 

The basic principle of our approach is the flooding of failure suspicion information over 
the network based on a local query-response mechanism. The algorithm proceeds execution 
by rounds. At each query-response round, a node broadcasts a query message to the nodes 
of its range until it possibly crashes. The time between two consecutive queries is finite but 
arbitrary. A QUERY message sent by a node includes two sets of nodes : the set of nodes 
that it currently suspects of being faulty, and a set of the mistakes i.e., the nodes that 
were erroneously suspected of being faulty previously. Each node keeps a counter, which is 
incremented at every round. Every new information that is generated by this node about 
failure suspicions or correction of false suspicions (mistakes) within a round is tagged with 
the current value of such a counter. This tag mechanism avoids old information to be taken 
into account by nodes of the network. 

Upon receiving a query message from a node of its range, a node sends it back a 
RESPONSE message. A query issued by a node is satisfied when it receives at least d — f 
corresponding RESPONSE messages. Moreover, each couple of query- response messages 
are uniquely identified in the systenjl. Notice that we assume that a node issues a new 
QUERY only after the previous one is terminated. Moreover, when a node broadcasts a QUERY 
message, we assume that it receives the query too, and that its own response always arrives 
among the first d — f responses it is waiting for. 

^For the sake of simplicity, such identification is not included in the code of the algorithms of the paper. 
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4.2 Behavioral Properties 

Let us define some behavioral properties that processes should liave in order to ensure 
that our proposed implementation of a failure detector satisfies the properties of class (^S 
in an unknown network. 

In order to implement any type of unreliable failure detector with an unknown member- 
ship, processes should interact with some others to be known. According to |FJA06j . if there 
is some process in the system such that the rest of processes have no knowledge whatsoever 
of its identity, there is no algorithm that implements a failure detector with weak complete- 
ness, even if links are reliable and the system is synchronous. Thus, in order to implement 
a '05' failure detector, the following membership property, namely MV, should be ensured 
by all processes in the system. 

Property 1. Membership Property {A4V). Let t E T. Denote known^^ the set of pro- 
cesses from which pj has received a QUERY message at time t. Let be the set of processes 
Pj that, at time t, have received a QUERY from pi. That is, — {pj \ pi G known j}. A 
process pi satisfies the membership property if : 

MV(j)i) '^^ 3t>QET ■.\Kl\> f + 1 

This property states that, to be part of the membership of the system, a process pi 
(either correct or not) should interact at least once with other processes in its range by 
broadcasting a query message. Moreover, this query should be received and represented in 
the state of at least one correct process in the system, beyond the process pi itself. 

Let us define another important property in order to implement a timer-free failure 
detector in a system with an unknown membership. It is the responsiveness property, namely 
TZV, which denotes the ability of a node to reply to a query among the first nodes. 

Property 2. Responsiveness Property (TZP). Let t,u eT. Denote rec^fronij the set 
of d — f processes from which pj has received responses to its QUERY message that terminated 
at or before t. The TZV property of the correct process pi is defined as follows : 

TZP{pi) 3?! e T : > u,'ipj G rangci, pi G rec-from^j 

Intuitively, the TZV{pi) property states that after a finite time u, the set of the d — f 
responses received by any neighbor of pi to its last query always includes a response from 
Pi- 

4.3 Implementation of a Failure Detector of Class <^S' for Unknown 
Networks 

Algorithm [1] describes our protocol for implementing a failure detector of class {}S when 
the underlying system is an f-covering network, satisfying the behavioral properties. 
We use the following notations : 
- counter i : denotes the round counter of node pi. 
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- suspectedi : denotes the current set of processes suspected of being faulty by pi. Each 
element of this set is a tuple of the form {id, counter) , where id is the identifier of the 
suspected node and counter is the value of counteri when pi generated the information 
that it suspected node id of being faulty. 

- mistakci : denotes the set of nodes which were previously suspected of being faulty but 
such suspicions are currently considered to be false. Similar to the suspectedi set, the 
mistakci is composed of tuples of the form {id, counter) i.e, counter indicates when 
the information that id is falsely suspected was generated. 

- rec-fromi : denotes the set of nodes from which pi has received responses to its last 
QUERY message. 

- knowni : denotes the current knowledge of pi about its neighborhood, knowni is then 
the set of processes from which pi has received a QUERY messages since the beginning 
of execution. 

- Add{set, {id, counter)) : is a function that includes {id, counter) in set. If an {id,—) 
already exists in set, it is replaced by {id, counter). 

The algorithm is composed of two tasks. Task Tl is made up of an infinite loop. At each 
round, a QUERY message is sent to all nodes of pi's range (linelH]). This message includes the 
set of nodes that pi currently suspects and the set of mistakes of which pi is aware. Node pi 
waits for at least d — f responses, which includes pi's own response (line[7|). Then, pi detects 
new suspicions (lines I MTSl) . pi starts suspecting each non previously suspected node pj that 
it knows {pj € knowrii) but from which it does receive a RESPONSE to its last query. If a 
previous mistake information related to this new suspected node exists in the mistake set 
mistakci, it is removed from it (line I12p and the counter counteri is updated to a value 
greater than the mistake tag (line fTTj) . The new suspicion information is then included in 
suspectedi with a tag which is equal to the current value of counteri (line fT4|) . Finally, at 
the end of task Tl, counteri is incremented by one fline llGp . 

Task T2 allows a node to handle the reception of a query message sent by another 
node of its range. A query message contains the information about suspected nodes and 
mistakes kept by the sending node. However, based on the tag associated to each piece of 
information, the receiving node only takes into account the ones that are more recent than 
those it already knows. 

The two loops of task T2 respectively handle the information received about suspected 
nodes (lines HTHSD) and about mistaken nodes flines [52HS7| . Thus, for each node Px included 
in the suspected (respectively, mistake) set of the query message, pi includes the node Px 
in its suspectedi (respectively, mistakci) set only if the following condition is satisfied : pi 
received a more recent information about Px status (failed or mistaken) than the ones it has 
in its suspectedi and mistakci sets. A more recent information is characterized by the fact 
that Px has never been suspected or false suspected by pi or by the fact that its counter in 
the Pi sets is less than the new received counter x (see lines and 155]) . In such a case, pi 
also removes the node Px from its mistakCi (respectively, suspectedi) set (lines and 155]). 

Furthermore, in the first loop, a new mistake is detected if the receiving node pi is 
included in the suspected set of the query message fline [^5|) . Then, pi adds itself in its local 
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Algorithm 1 Asynchronous Implementation of a Failure Detector 

1: init : 

2: suspectedi *— 0; mistakci «— 0; counteri <— 

3: knowrii <— 

4: Task Tl : 

5: loop 

6: broadcast QUERY(sMspectedi, mistakci) 

7: wait until RESPONSE received from at least (d — /) distinct processes 

8: rec^frorrti <— tlie set of distinct nodes from wliicli pi has received a response at line [71 

9: for all pj £ knowrii \ rec^frorrii \ {pj, — ) suspectedi do 

10: if {pj , counter) G mistakci then 

11: counteri = max(counteri, counter + 1) 

12: mistakci = mistakci \ {pj, — ) 

13: end if 

14: Add(suspectedi, {pj , counteri)) 

15: end for 

16: counteri = counteri + 1 

17: end loop 

18: Task T2 : 

19: upon reception of QUERY {suspected j, mistake j) from pj do 

20: knowni <— knowni U {pj } 

21: for all {px,counterx) £ suspectedj do 

22: if (pa;, — ) suspectedi U mistakci or {px, counter) £ suspectedi U mistakci \ counter < counter^ 
then 

23: if Pa; = Pi then 

24: counteri = max(counteri, counterx + 1) 

25: Addimistakci, {pi, counteri)) 

26: else 

27: Add{suspectedi , {p^, counterx)) 

28: mistakci = mistakci \ {px, —) 

29: end if 

30: end if 

31: end for 

32: for all {px, counterx) £ mistake j do 

33: if {px, —) suspectedi U mistakci or {px, counter) £ suspectedi U mistakci \ counter < counterx 
then 

34: Add{mistakei, {px, counterx)) 

35: suspectedi = suspectedi \ {Px, —) 

36: end if 

37: end for 

38: send RESPONSE to pj 



mistake set (line I25p . The tag counteri associated to this mistake is equal to the maximum 
of the current value of counteri and the tag associated to the suspicion of pi, included in 
suspectedj set, incremented by one fline [24|). At the end of task T2 (line [38]) . pi sends to 
the querying node a response message. 
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4.4 Example of the Execution of the Algorithm 

Figure [1] illustrates an execution which shows the strong completeness property of Al- 
gorithm [T] We consider an 1-covering network (/ = 1) whose range density is equal to 3. 
Thus, each querying node should wait for at least 2 responses (one from itself and the other 
from one of its neighbors). 



knowrig^ {A,B,C,D} 
B D 




C E 
known = {A,B,C,E} 




(a) 



suspected^={<A,5>} 
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B D 
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suspected^={<A,10>} 
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suspectedg={<A,10>} 
B ^ D 
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suspected^={<A,10>} 

suspected|^={<A,10>} 
(d) 

Fig. 1 - Example of Failure Detection 



C E 
suspectedj,={<A, 1 0>} 

suspected|^={<A,10>} 

(c) 



We do not show a scenario from the beginning of execution of the algorithm, but one 
where every node i is already aware of the participants of its range (knowrii), see step (a). 
In step (b), A fails. Thus, as neither node B nor node C receive a responses from A to 
their respective QUERY, they start suspecting A. At the moment of the query counterB 
is equal to 5 (see suspectedB) but counterc is equal to 10 (see suspectedc)- Then, both 
B and C propagate their suspected sets to their neighbors in their next respective QUERY 
rounds as shown in step (c). Nodes D and E will include the corresponding information 
{A, 5) and {A, 10) in their respective sets suspectedo and suspectedE- Node B will update 
its suspectedB set since the counter of the received information from C is greater than the 
one that it keeps in its suspectedB- However, C will discard the information received from 
B. Similar to step (c), in step (d) nodes i?, C, D and E include in their next QUERY message 
their respective suspected sets. Therefore, eventually the information {A, 10) related to the 
failure of A will be delivered to all correct nodes of the network. 



INRIA 



Failure detector for dynamic networks 



11 



4.5 Proof 

We present in this section a sketch of proof of both the strong completeness and eventual 
weak accuracy properties of our algorithm that characterize failure detectors of class (^S for 
an f-covering network composed of non-moving nodes. 

Consider that the most recent status about a process Px is stored in a supected or mistake 
set and represented by the tuple (p^, ct^) which has the greatest counter ct^ in the network. 
In case of equality between a suspicion and a mistake, we give arbitrarily precedence to the 
mistake. 

Lemma 1. Consider an f-covering network. Let pi be a correct process. Consider that, at 
time t, Pi owns the most recent status about px in the network ({px,ctx)) in its suspectedi 
set (respectively, mistakci set). If no more recent information about px status is genera- 
ted afterward, then eventually all correct nodes will include {px,ctx) in their suspected set 
(respectively, mistake set). 

Demonstration. Since pi is correct, it will execute line El and broadcast a query message 
containing (pxTctx) in the suspectedi set (respectively, mistakci set) to all its neighbors. As 
channels are reliable, this query message is received by every correct process pj € rangei. 
Thus, Pj will execute lines [5l][3l] (respectively, lines 15^11571) . Since ctx is the greatest counter 
associated with px in the network, pj executes line [57] (respectively, line l34p and add (px, ctx) 
to its own suspected j set (respectively, mistake j set). In the next round, pj, the same as pi, 
must broadcast this new status regarding p^; in its respective sets. Thus, due to the f-covering 
network property, all nodes in the network eventually add {px,ctx) in their suspected set 
(respectively, mistake set) and the lemma follows. □ 

Lemma 2. Consider an f-covering network in which all processes satisfy AiV. Let pf be 
a faulty process. If process pi is correct then eventually pf is permanently included in its 
suspectedi set. 

Demonstration. Let us consider that pf crashes at time t. 

Remark 1. Since A4V{pf) is satisfied, pf has sent to processes in range/ at least one query 
message before it crashed at time t. Then, a number of correct processes within range f will 
include p/ in their respective known set which is updated when a process receives a query 
(line [50)) . Let us denote K this set of processes. Notice that, by AiV, \K\ > / + 1, and then 
there is at least one correct process pi such that p/ G knowni. 

Remark 2. As pf has crashed, there will be a time t' > t after which all processes in 
K will never receive a response message from pf (i.e., Pf ^ rec-from sets of processes 
within K) (line [7|). Thus, if p/ was not already suspected by these processes (line [9]), it 
will be included in their corresponding suspected sets with a tag equal to the current value 
of their respective counter or with a greater tag then the one associated with p/ in the 
mistake set if it was previously in there (line I14p . At this point no more information about 
Pf can be generated since only pf can generate a mistake about itself (line I23p and only 
processes in K can generate a new suspicion and p/ is already in their suspected set. Thus, 
the most recent information about pf sent in a query message is either (1) a suspicion 
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or (2) a mistake. In the first case, following Lemma [Tl all correct processes will eventually 
include pf in. their respective suspected set. Since no new information about p/ is generated, 
Pf is permanently suspected by all correct nodes. In the second case, by Lemma [U the 
mistake eventually reach a correct process pi in which removes pf from suspectedi. At 
the next round, pi will include Pf in suspectedi with a greater tag since Pf ^ rec^frorrii 
and Pf ^ suspectedi. This information will in turn be propagated to all correct processes, 
following the propagation Lemma[T] Thus, all correct processes will permanently suspect pf 
since no new information about Pf is generated. □ 

Lemma 3. Consider an f-covering network in which all processes satisfy MV. Let pi be a 
correct process which satisfies the responsiveness property TZV^Pi). There is a time u after 
which Pi is not included in the suspected j set of any correct process pj . 

Demonstration. Remark 1. According to TZV{pi), there is a time t after which every process 
Pj in the neighborhood of pi receives a RESPONSE message from pi in reply to their query. 
Thus, after time pi is always included in the rec-from sets of all nodes within its rangci. 
Since a process starts being suspected only if its reply is not received by one of its neighbor 
(lines [9lfT5|) . no process adds pi to its suspected set due to a query message sent after time 
t. 

Remark 2. If pi is not included in any suspected set in the network, clearly pi cannot 
be suspected anymore. If pi is included in at least one suspected set, there are two cases 
to consider : the most recent piece of information about pi is either (1) a mistake or (2) 
a suspicion. In the first case, based on Lemma [l] all processes which were suspecting pi 
will eventually execute lines I34II35I upon receiving the propagated mistake and remove pi 
from their suspected set definitely. In the second case, following Lemma[TJ pi will eventually 
deliver a query message with pi in the suspected set. This will cause pi to generate a new 
mistake with a greater tag. This mistake will in turn be propagated to all processes, which 
will remove pi from their suspected set if they were suspecting it. □ 

Theorem 1. Algorithm]^ implements a failure detector of class (}S, assuming an f-covering 
network of non-moving nodes which satisfies the behavioral properties TZV, M-V and with 
f <n. 

Demonstration. Consider a correct process pi and a fault process pf. To satisfy the strong 
completeness property, we must prove that eventually^/ is permanently included in .suspectedi 
set Pi. This claim follows directly from LemmaHl To satisfy the eventual weak accuracy pro- 
perty, we must prove that there is a time u after which pi is not included in the suspectedj 
set of any correct process pj . This claim follows directly from Lemma [3] and the theorem 
follows. □ 
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5 Extension for Mobility Management 

In this section we present an extension for Algorithm [T] that supports mobihty of nodes. 
For such an extension some new behavioral properties in respect to mobility of nodes and 
the underlying system must be defined. 

5.1 Mobility Behavioral Properties 

Let pm be a moving node. Notice that a node can keep continuously moving and recon- 
necting, or eventually crashes. Nonetheless, we consider that pm should stay connected to 
the network for a sufficient period of time in order to be able to update its state with recent 
information regarding failure suspicions and mistakes. Otherwise, it would not update its 
state properly and thus completeness and accuracy properties of the failure detector would 
not be ensured. Hence, in order to capture this notion of "sufficient time of reconnection" , 
the following mobility property, namely MobiV, has been defined. 

Property 3. Mobility Property (MobiV). Let t £ T. Let Q\ be the set of processes from 
which Pi has received a query message that terminated before or at t. A process pi satisfies 
the mobility property if : 

Mob^V{p^) ''^■'^ 3i > e T : |Q*| > / + 1 

This property should be satisfied by all moving nodes when they reconnect to the net- 
work. Thus, MobiV{pm) ensures that, after reconnecting, there will be a time at which 
process Pm should have received query messages from at least one correct process, beyond 
itself. Since query messages carry the state of suspicions and mistakes in the membership, 
this ensures that process p„i will update its state with recent informations. 

We assume also that the membership property holds for all moving nodes when they 
reconnect to the network. Thus, MV{pm) ensures that, after reconnecting, there will be 
a time at which process Pm interacts at least once with other processes in its rangem, 
broadcasting a query message which will be delivered by at least one correct processes in 
rangem, beyond pm. 

Regarding the underlying system behavior, we consider that despite mobility, the /- 
covering property of the network is ensured and that the range density d of the network 
does not change. Moreover, we have extended the TZV property such that neighbors of a 
node p, which has the TZV property, eventually stop moving outside p's range. Otherwise, 
even if p has the TZV property, a moving node would add p in its known set whenever it 
belonged to p's range and then it would suspect p when it moved outside p's range. The 
extension of TZV property, namely MobiTZV, is defined as follows : 

Property 4. Mobility Responsiveness Property (AiobiTZV) . Let t G T. Denote range\ 
the set of processes in rangci at t. A process pi satisfies the mobility responsiveness property 

MobiTZVip.,)'^^^ TZP{p,) -.BueT -.yt > u,W > t , Pj G range\ =^ pj G range\ 
MobiTZV should hold for at least one correct non-moving node. 
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5.2 Implementation of a Failure Detector of Class (}S for Mobile 
Unknown Networks 

The extension of the algorithm to support mobihty of nodes is based on the same query- 
response principle presented in Section [H When a node Pm moves to another range, it 
starts being suspected of having crashed by those nodes of its old range, since it cannot 
reply to query messages from the latter anymore. Hence, query messages that include 
Pm as a suspected node will be propagated to nodes of the network. Eventually, when 
reconnects to the network, it will receive such suspicion messages. Upon receiving them, pm 
will correct such a mistake by including itself (pm) in the mistake set of its corresponding 
QUERY messages. Such information will be propagated over the network. On the other hand, 
Pm will start suspecting the nodes of its old range since they are in its known set. It then will 
broadcast this suspected information in its next query message. Eventually, this information 
will be corrected by the nodes of its old range, and the corresponding generated mistakes 
will spread over the network, following the same principle. Notice that, in order to avoid a 
"ping-pong" effect between information about failure suspicions and corrections (mistakes), 
a mechanism should be added to the algorithm in order to remove from known sets those 
nodes that belong to remote ranges. 

In Algorithm!!! we just show the lines which need to be included in task T2 of Algorithm 
[T]in order to support mobility of nodes. Lines should be added in the if block of the 

second loop of task T2, just after line [35] of Algorithm [1] They allow the updating of the 
known sets of both the moving node pm and of those nodes that belong to the original range 
oi Pm- For each mistake {px, counter^) received from a node pj such that node pi keeps an 
old information about p^, Pi verifies whether p^ is the sending node pj. In they are different, 
Px should belong to a remote range rangcx, such that px ^ rangei. Thus, process Px is 
removed from the local set knowni. 



Algorithm 2 Asynchronous Implementation of a Failure Detector with Mobility of Nodes 

36: if {px 7^ Pj) then 

37: knowrii = knowui \ {px} 
38: end if 



5.3 Proof 

We present in this section a sketch of proof of both the strong completeness and eventual 
weak accuracy properties of the extended algorithm [2] that characterize failure detectors of 
class (}S for an f-covering network composed of moving and non-moving nodes. 

Lemma 4. (1) Infinitely often, during a run, the knowni set contains either correct pro- 
cesses which are in rangCi or faulty processes. Moreover, (2) for every process pi which 
satisfies M'P{pi), then there is a correct process pj, such that pi G known j. 
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Demonstration. Let us observe that the QUERY-response messages are exchanged between 
processes in the same range. Thus, on the execution of hne [2D1 the set knownt is updated 
when Pi receives query messages from other processes in its rangci. Beyond hne l20l knowui 
may be updated at hues I36II381 in order to remove nodes suspected to be in another range, 
different from p^'s range. This may happen due to a mobihty. Thus, if a process which raised 
a mistake (px) is different from the process who carries it (pj), probably does not belong 
to rangci, because otherwise, pi would have received the mistake hy Px itself. It may happen 
that Px was in rangci at some point in time, but due to a move, it has changed to another 
neighborhood, such that Px ^ rangci. Wherever the case, process pi is going to remove Px 
from its knowUi set and the part (1) of this lemma follows. 

Let us prove part (2) of the lemma. Since MV{pi) is satisfied, there is at least one 
correct process pk which has received a QUERY message from pi after pi has connected or 
reconnected to the network at time t. Thus, pi G knowuk- Nonetheless, later, pi can be 
removed from knowUk by the execution of lines I36H38I due to a suspicion of mobility. But, 
notice that, since channels are reliable, the query from pi in which pi 6 mistakci is going 
to eventually arrive iopk- In this case, two situations can occur. Situation (1). If this QUERY 
is the first one to arrive at pk, it will satisfy the predicate of line [551 thus lines are 
executed, but not lines [55^[551 Afterward, when a QUERY from a process pj arrives containing 
the mistake over pi , and such that pi ^ pj , then since this mistake has already been taken 
into account, the predicate of hne [33] will not be satisfied and lines [36H381 are not executed. 
Thus Pk will not remove pi from knowuk set. Situation (2). A query from a process pj is 
the first one to arrive at pk containing the mistake over pi, and such that pi ^ pj. In this 
case, the predicate of line [33] is satisfied and lines [55H551 are executed. Thus pk removes pi 
from knowuk- Nonetheless, later, a query from pi arrives in which pi £ mistakei. In this 
case, process pk will execute line 1201 including pi in knowUk- Moreover, since this mistake has 
already been taken into account, the predicate of line [551 will not be satisfied and lines [55H551 
are not executed. Thus pk will not remove pi from knowuk set. This concludes the proof of 
part (2). 

□ 

Lemma 5. Consider an f-covering network in which all nodes satisfy A4V and all moving 
nodes satisfy MobiV. Lemma\^holds for every correct process pi (moving or non-moving) . 

Demonstration. The lemma follows directly from Lemma [1] if pi is a non-moving node. To 
take into account moving nodes, we should consider two cases. Case (1). Assume that pi is 
a correct moving node which has the most recent status about process Px- As soon as pi 
reconnects to the network at time i', it will execute line [H] and broadcast a query message 
to all its neighbors. Since A4V{pi) holds, pi is correct and channels are reliable, every correct 
node Pj € rangei receives this query message. Since, \rangei\ > / + 1, there will be at least 
one correct non-moving node pk which receives this query. Thus, by the same arguments 
of Lemma [TJ the lemma follows. 

Case (2). Assume that pi is a correct moving node which has not yet the most recent 
status about process Px and let us consider that due to Lemma [U every non-moving node 
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has added px in its suspected (respectively, mistake) set before or at time t. As soon as 
Pi reconnects to the network at time t' > t, since M.ohiV{pi) is satisfied, pi wiU receive 
QUERY messages from at least a correct process pj with the last status of suspicion and mis- 
taken informations about Px- Thus, pi will eventually add Px in its suspectedi (respectively, 
mistakei) set and the lemma follows. 

□ 

Lemma 6. Consider an f-covering network in which all nodes satisfy A4V and all moving 
nodes satisfy AdobiV. Letpf be a faulty process (moving or non-moving). If process pi (mo- 
ving or non-moving) is correct then eventually pf is permanently included in its suspectedi 
set. 

Demonstration. If pi and p f are non-moving nodes, the lemma follows directly from LemmaO 
To take into account moving nodes, let us assume that pi is a correct moving node which 
has the most recent status about process p / . Due to Lemma and the same arguments of 
Lemma [5] (Remark 1), p/ is in the known set of at least one correct process in the network. 
We should consider the following cases. 

Case (1). Consider that pf crashes at time r < t. Let us suppose that pi is the only 
correct process such that pj € knowui. Moreover, before broadcasting this information to its 
neighborhood, pi moves at time t. Since pi keeps its state during the moving, p/ G suspectedi. 
When Pi reconnects to the network at time t' , due to Lemma [SJ this information about the 
suspicion Pf will be propagated to all correct nodes in the network. Finally, due to the 
same arguments of Lemma [2] (Remark 2) and LemmaEJ p/ is permanently included in every 
suspected set of a correct process, either moving or non-moving. 

Case (2). Consider that pf crashes at time s, t < s < t'. Suppose that pi has pf in 
its mistakci when it starts moving at time t. Since pi keeps its state during the moving, 
Pf (z mistakci when pi reconnects to the network at time t' . Since pi has the most recent 
status about p/, then, due to Lemma this information about the mistake of will be 
propagated to all correct nodes in the network. Nonetheless, as soon as pf is faulty, due 
to the same arguments of Lemma [2] and Lemma [SJ p/ is permanently included in every 
suspected set of a correct process, either moving or non-moving. 

□ 

Lemma 7. Consider an f-covering network in which all nodes satisfy AdV and all moving 
nodes satisfy AiobiV . Let pi be a correct non-moving node which satisfies the mobility res- 
ponsiveness property MobiTZVipi). There is a time u after which pi is not included in the 
suspectedj set of any correct process pj (moving or non-moving). 

Demonstration. Since AiobiTZV{pi) is satisfied, there is a time s after which, TZV{pi) holds 
and nodes in the neighborhood of pi do not leave rangci. Thus, due Lemma[3] (Remark 1), 
there is a time s' after which, no process in the network adds pi to its suspected set (on to 
the execution of lines [MTK|l . 

Due to Lemma[2] (Remark 2), we can ensure that pi will not be included in any suspected 
set of non-moving correct nodes. We must then prove that eventually pi is not included in 
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the suspectedm set of any correct moving node Pm- Let us consider a correct moving node 
Pm starting to move at time t and stopping to move at time t' . Notice that, if pm does not 
suspect Pi before moving at time t, the claim foUows from Lemma [3] (Remark 2). Suppose 
that Pm suspects pi before or at time t. Then, since pm keeps its state during the moving, 
Pi € suspectedm when pm reconnects to the network at time t' . If the suspicion over pi 
represents the most recent information in the network, due to Lemma [SJ it is going to be 
diffused to aU correct nodes. Nonetheless, as soon as pi is correct, pi will revoke such a 
suspicion by the execution of lines [23ll26t which will generate a new mistake with a greater 
tag. Due to Lemma [SJ this mistake will be propagated to all correct processes, then pm will 
permanently remove pi from its suspectedm set. 

□ 

Theorem 2. Algorithm\^ implements a failure detector of class (}S, assuming an f- covering 
network of moving and non-moving nodes which satisfies the behavioral properties TZV, MV, 
MohiV and MobiUP. 

Demonstration. The strong completeness property follows directly from LemmaEl The eventual 
weak accuracy property follows directly from Lemma[7]and the theorem follows. □ 

6 Performance Evaluation 

In this section we study and evaluate the behavior of our asynchronous failure detector 
compared to a timer-based one. To this end, we have chosen the gossip-based heartbeat 
unreliable failure detector proposed by Friedman and Tcharny in pT05j . 

Our performance experiments were conducted on top of the OMNeT-|--t- discrete event 
simulator [omnj . We assume a two-dimensional region S of 700TOx700m. Transmission range 
r is set to 100m in all runs. The number of nodes N is fixed to 100 and each simulation lasts 
30 minutes. The one-hop network delay S is equal to 1ms in average. Since our unreliable 
failure detector needs a network where the f^covering property always holds, the N nodes 
can not be placed randomly inside the region S. The initial topology of the network is in 
fact gradually built before the beginning of execution of an experiment. Thus, we start by 
inserting a graph clique of / -I- 2 nodes organized in a circle whose radius is equal to r/2. 
Then, at each step, a new node of S is randomly chosen. The latter is included in the 
network regardless it has / -1- 1 neighbors in the current configuration. The construction of 
the network stops when it reaches N nodes. 

In the unreliable FD proposed by Friedman and Tcharny, a node periodically sends 
heartbeat messages to its neighbors. A vector is included in every heartbeat message such 
that each entry in the vector corresponds to the highest heartbeat known to be sent from 
the corresponding node. Every A time units, each node increments the entry of the vector 
corresponding to itself and then broadcasts its heartbeat to its neighbors. Based on the 
performance experiments described in the authors's article, we have set A to Is. Upon 
receiving a heartbeat message, a node updates its vector to the maximum of its local vector 
and the one included in the message. A node also associates a timer to each other node of 
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the system. Thus, node j set the timer of i to O whenever it receives a new information 
about i. On the other hand, if the timeout of i expires, it is considered suspected by j. Note 
that the value of O should take into count higher communication delay due to longer paths 
between two nodes. We have set the value of 9 to 2s. 

Concerning the implementation of our FD, it is not feasible that a node continuously 
broadcasts a query message since the network would be overloaded with messages. To 
overcome this problem, we have included a delay of A units of time between lines [7] and |H] 
of the Algorithm [TJ Similar to the Friedman and Tcharny's approach, we have set A to Is. 
However, by adding this waiting period, a processes may receive more than d — f replies. 
Therefore, the extra replies will also be included in the rec-from set of this process (line[S]), 
reducing then the number of false suspicions. It is worth remarking that this improvement 
does not change the protocol correctness. 

6.1 Failure Detection 

In order to evaluate the completeness property of both failure detectors, we have mea- 
sured the impact of the range density d of the network on their respective failure detection 
time (Figure 12). The number of faults is equal to 5 and they are uniformly inserted during 
an experiment. The range density d varies from 7 to N/2 nodes. For each density, we have 
measure the average, maximum and minimum failure detection time. 
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Fig. 2 - Failure detection time vs. density 

We observe that for both failure detectors there is no false suspicion. Furthermore, the 
propagation of failure suspicions is quite fast because the diameter of the network is relatively 
small. In the case of Friedman and Tcharny's FD, the mean failure detection time is always 
between — A and Q time units, independently of d since failures are detected based on 
heartbeat vector values and timers. Such limit values can be explained : if node i crashes 
just after node j has set its timer related to i to Q, j will detect the crash of i after 8 units 
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of time; if i crashes just before broadcasting a heartbeat, i.e. just after A units of time, 
j wiU detected the crash of i after — A units of time. On the other hand, for our FD, 
the failure detection time decreases with the range density. This happens because failure 
detection information is included in query messages which spreads faster over the network 
when the density increases. We can notice that for values of d greater than 22, the failure 
detection time is uniform and equals around A + 5. 

The maximum failure detection time characterizes the time for all nodes to detect a 
failure (strong completeness). We can observe that compared to Friedman and Tcharny's 
FD, this time is smaller and homogeneous for our FD, which can be also explained by the 
above mentioned propagation of failure information in query messages. 

6.2 Impact of mobility 

We have evaluated the accuracy property when a node m which has 7 neighbors and is 
located at one boundary of the network moves about 500m at a speed of 2m/s. It starts 
moving at time IGOs. We consider that while moving, node m does not interact with the 
other nodes as if it travels through a disturbance region where it can not send or receive 
any message. Thus, m stops executing while it moves. Furthermore, all neighbors of m must 
have d — f + 1 neighbors. Such restriction is necessary to guarantee that at least d — f nodes 
will reply to the query of these old neighbors of m after it moves. The range density d of 
the network is equal to 7 and there is no fault. 
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Fig. 3 - Total number of false suspicions 

For each experiment, the total number of false suspicions has been measured. Figure [3] 
shows the moment just before and after node m stops moving at time 356s. We can observe 
that all A'^ — 1 nodes suspect m before this time in both failure detectors. After it, false 
suspicions about node m start being corrected by all nodes. In Friedman and Tcharny's FD, 
there are no more false suspicions in around 1.5s. False suspicions about node m will also 
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start being corrected in our FD since m generates a mistake which is propagated over the 
network. However, node m at the same time starts suspecting its 7 old neighbors. Thus, it 
broadcasts such suspicions in its next query message. This information spreads over the 
network and nodes of the system wiU start suspecting them too. This is the reason why the 
total number of false suspicions starts increasing after 357s till 358s when almost all nodes 
suspect the 7 old neighbors of m. However, at this time such an information also reaches the 
latter that then generate the corresponding mistakes and broadcast them. Such mistakes 
are propagated to all nodes of the network. All false suspicions are corrected by all nodes at 
359.5s. 

7 Related Work 

As in our approach, some scalable failure detector implementations do not require a fully 
connected network. Larrea et al. proposed in [LFAOOj an implementation of an unreliable 
failure detector based on a logical ring configuration of processes. Thus, the number of 
messages is linear, but the time for propagating failure information is quite high. In [ GCGOl] . 
Gupta et al. proposed a randomized distributed failure detector algorithm which balances the 
network communication load. Each process randomly chooses some processes whose aliveness 
is checked. Practically, the randomization makes the definition of timeout values difficult. In 
|BMS03j . a scalable hierarchical failure adapted for Grid configurations is proposed. However, 
the global configuration of the network is initially known by all nodes. It is worth remarking 
that none of these works tolerate mobility of nodes. 

Few implementations of unreliable failure detector found in the literature focus on MA- 
NET environments. All of them are timer-based ones. In the Friedman and Tcharny algo- 
rithm |FT05| . authors assumes a known number of nodes and that failures include message 
omissions too. In jTTS04j . the authors exploit a cluster-based communication architecture 
for implementing a failure detector service able to support message losses and node failures. 
However, they provide probabilistic guarantees for the accuracy and completeness properties. 

Sridhar presents in [Sri06j the design of a hierarchical failure detection which consists of 
two independent layers : a local one that builds a suspected list of crashed neighbors of the 
corresponding node and a second one that detects mobility of nodes across network, which 
corrects possible mistakes. Contrarly to our approach that allows the implementation of FD 
of class OS*, the author's failure detector is an eventually perfect local failure detector of 
class 0-P i-e., it provides strong completeness and eventual strong accuracy but with regard 
to a node's neighborhood. 

In order to solve the problem of reaching agreement in mobile networks where processes 
can crash, Gavin et al. [CSS05j have adapted the failure detector definition of [GT96| to 
the case where the participants are unknown. They have introduced the concept of local 
participant detectors, which are oracles that inform the subset of processes that participating 
in the consensus. The authors construct an algorithm that solves consensus with an unknown 
number of participants in a fail-free network. Furthermore, they extend their solution and 
prove that a perfect failure detector {V) is required for solving the fault-tolerant consensus 
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with a minimum degree of connectivity. Greve et al. |GT07] have subsequently extended this 
work, by providing a solution for the consensus in a fail-prone network which considers the 
minimal synchrony assumption (i.e., the ^S), but at the expenses of requiring a higher 
degree of connectivity involving with the set of participants. We believe that our proposed 
()S FD will be of great interest to implement this consensus algorithm over a MANET. 



8 Conclusion 

This paper has presented a new implementation of an unreliable failure detector for 
dynamic networks such as MANETs, where the number of nodes is not initially known and 
the network is not fully connected. Our algorithm is based on a query-response mechanism 
which is not timer-based. We assume that the network has the f-covering property, where 
/ is the maximum number of failures. This property guarantees that there is always a path 
between two nodes despite of failures. Our algorithm can implement failure detectors of 
class (}S when both the behavioral responsiveness {TZV, MobiTZV), membership {M.V) and 
mobility (AdobiV) properties are satisfied by the underlying system. The proposed algorithm 
supports mobility of nodes as well. As a future work, we plan to adapt our algorithms and 
properties to implement other classes of failure detectors. 
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